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1.  Introduction 


Let  (IT,  SF,  P )  be  a  probability  space  with  points  co  £  IT  and  let  ( yn ,  ^n), 
n  =  1,  2,  •  •  •  ,  be  an  integrable  stochastic  sequence:  yn  is  a  sequence  of  random 
variables,  $ n  is  a  sequence  of  cr-algebras  with  &n  C  SFn+i  C  yn  is  measurable 
with  respect  to  $n,  and  E(yn )  exists,  —  oo  ^  E(yn)  ^  <».  A  random  variable 
s  =  s(co)  with  positive  integer  values  is  a  sampling  variable  if  {s  ^  n}  £  Tn  and 
{$  <  oo }  =  IT.  (We  denote  by  {•  *  *}  the  set  of  all  co  satisfying  the  relation  in 
braces,  and  understand  equalities  and  inequalities  to  hold  up  to  sets  of  P-meas- 
ure  0.)  We  shall  be  concerned  with  the  problem  of  finding,  if  it  exists,  a  sampling 
variable  s  which  maximizes  E(y8). 

To  define  a  sampling  variable  $  amounts  to  specifying  a  sequence  of  sets 
Bn  €=  $n  such  that 

(1)  0  =  Bo  C  •  •  •  C  Bn  C  Bn+ 1  C  •  •  •  ;  U  Bn  =  IT, 

i 

the  sampling  variable  s  being  defined  by 

(2)  {s  S  n}  =  Bny  {s  =  n}  =  Bn  —  Bn-\. 

We  shall  be  particularly  interested  in  the  case  in  which  the  sequence  (yn,  SFn) 
is  such  that  the  sequence  of  sets 

(3)  Bn  =  {E(yn+i\5n)  ^  yn} 

satisfies  (1).  We  shall  call  this  the  monotone  case .  In  this  case  a  sampling  vari¬ 
able  s  is  defined  by 

(4)  {s  g  n)  =  {E{y n+i\5n)  ^  yn}, 


and  s  satisfies 

,  f  >  yn, 

(5)  E(yn+l\5n){ 

=  Vny 


s  >  ny 
s  g  n. 


The  relations  (5)  will  be  fundamental  in  what  follows. 
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In  the  monotone  case  we  have  for  the  sampling  variable  $  defined  by  (4)  the 
following  characterization : 

(6)  5  =  least  positive  integer  j  such  that  E(yJ+ i^-)  ^  yjt 

Now  even  in  the  nonmonotone  case  we  can  always  define  a  random  variable  s 
by  (6),  setting  s  =  °c  if  there  is  no  such/;  let  us  call  it  the  conservative  random 
variable.  The  following  statement  is  evident:  the  necessary  and  sufficient  con¬ 
dition  that  there  exists  a  sampling  variable  $  satisfying  (5)  is  that  we  are  in  the 
monotone  case,  and  in  this  case  $  is  the  conservative  random  variable. 

In  section  3  we  are  going  to  show  that  in  the  monotone  case ,  under  certain 
regularity  assumptions ,  the  conservative  sampling  variable  s  maximizes  E(ys). 


2.  An  example 

Before  proceeding  with  the  general  theory  we  shall  give  a  simple  and  instruc¬ 
tive  example  of  the  monotone  case  in  the  form  of  a  sequential  decision  problem. 

Let  x)  Xi,  x2)  •  •  •  be  a  sequence  of  independent  and  identically  distrib¬ 
uted  random  variables  with  E(x+)  <  co,  where  we  denote  a+  —  max  (a,  0), 
ar  =  max  (  —  a,  0).  We  observe  the  sequence  xi,  x2,  •  •  •  sequentially  and  can  stop 
with  any  n  ^  1.  If  we  stop  with  xn  we  receive  the  reward  mn  =  max  (xi,  •  •  •  ,  xn), 
but  the  cost  of  taking  the  observations  xh  •  •  •  ,  xn  is  some  strictly  increasing 
function  g(n)  ^  0,  so  that  our  net  gain  in  stopping  with  xn  is  yn  =  mn  —  g{n). 
The  decision  whether  to  stop  with  xn  or  to  take  the  next  observation  xn+i  must 
be  a  function  of  Xiy  •  *  •  ,  xn  alone.  Problem :  what  stopping  rule  maximizes  the 
expected  value  E(y8),  where  s  is  the  random  sample  size  defined  by  the  stopping 
rule?  We  assume  that  the  distribution  function  F(u)  =  P{x  g  u}  is  known. 
That  E(yn)  exists  follows  from  the  inequality 

(7)  yi  1+  +  •  •  •  +  %n  ) 
which  implies  that  E(y£ )  <  oo . 

Let  be  the  a-algebra  generated  by  xh  •  •  •  ,  xn.  Then  (; yn ,  SF»)  is  an  integrable 
stochastic  sequence,  and  we  have 

(8)  E(yn+i\$n)  -  yn  =  j  [%i  -  mn]  dF(xn+ 1)  -  [g(n  +  1)  -  g(n)} 

=  J  (x  -  mn)+dF(x)  — /(n), 

where  we  have  set 

(9)  f(n)  =  g(n  +  1)  —  g(n)  =  cost  of  taking  the  (n  +  l)st  observation. 

Since  we  have  assumed  g(n)  to  be  strictly  increasing,  and  f(n)  >  0,  it  is  easily 
seen  that  there  exist  unique  constants  an  such  that 

(10)  j  {x  —  OLn)+  dF(x)  =  /(ft),  ft  ^  1. 


By  (8)  and  (10), 
(11) 
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K(yn+i\5n) 


f  >  Vn 

Vn 


if 

if 


01  n  ^  &ny 


7Yln  ^  Otn. 


The  conservative  random  variable  s  defined  by  (6)  is  therefore 
(12)  s  =  least  positive  integer  j  such  that  my  ^  ay. 

We  are  in  the  monotone  case  if  and  only  if  this  s  is  a  sampling  variable  and  for 
every  n 


(13)  {E(yn+l\5n)  g  Vn)  C  {E(yn+i\5n+l)  ^  yn+i}, 


that  is,  mn  ^  a„  implies  mn+i  2:  an+i,  which  will  certainly  be  the  case,  since 
m„  g  mn+ 1,  if  a„  ^  an+i,  that  is,  if  /(n)  is  nondecreasing  and  hence  a»  is  non- 
increasing .  M  e  s/iaZi  henceforth  assume  this  to  hold .  We  shall  now  show  that  in 
this  case  the  conservative  random  variable  s  is  in  fact  a  sampling  variable,  that 
is,  that  P{s  <  00}  =  1.  We  have 


(14)  {s  >  n}  =  {mn  <  a„} , 
and  hence 

(15)  P{s  <  00}  =  1  —  limP{$>  n}  =  1  —  lim  P{mn  <  an} 


^  1  —  lim  P{mn  <  «i}  =  1  —  lim  Pn{x  <  ai}  =  1, 

n  n 

since  by  hypothesis  /( 1)  >  0  so  that  by  (10),  P{x  <  aj  <  1.  In  fact,  for  any 
r  ^  0, 


(16)  E(sr)  =  £  nrP{s  =  n}  ^  J2  nrP{s  >  n  —  1} 

n  =  1  n  =  1 

^  1  +  E  nrP{mn-\  <  on} 

n=2 


=  1  +  J2  nrPn~l{x  <  ai}  <oo, 

n=  2 

so  that  s  has  finite  moments  of  all  orders. 

It  is  of  interest  to  consider  the  special  case  g(n)  =  cn,  0  <  c  <  oo.  Here 
f(n)  =  c  and  a„  =  a,  where  a  is  defined  by 

(17)  f  (*  -  «)+  <«?(*)  =  c, 

and  s  is  the  first  j  ^  1  for  which  ^  a.  Hence 
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(18) 


so  that 


P{s  =  j}  =  P{x  =  <  a}, 


= 


1 

P{x  ^  a}’ 


^(ffr)  =  L  =  j}E(™J  ~  cj\a  =  /), 
i-i 

2J(my|s  =  /)  =  Ufolsi  <«,•••,  Xy_i  <  a,  Zy  ^  a) 

=  /J  {x  ^  a}  /  * 

{*£«} 


(19)  =  P{xL  a-}  [  /  *  -  c] 

=  p{x^  a}-  [f  (*  -  «)+  W  -  c  +  «P{X  *  a}] 
an  elegant  relation. 


<*> 


3.  General  theorems 

In  the  following  three  lemmas  we  assume  that  ( yn ,  $Fn)  is  any  integrable 
stochastic  sequence  and  that  $  and  t  are  any  sampling  variables  such  that  E(y8) 
and  E(yt)  exist. 

Lemma  1.  If  for  each  n , 


(20) 

^(y.lffn)  1  yn 

if  $  >  n; 

and 

(21) 

E{yt\5n)  ^  yn 

if  s  =  n,  t  >  n, 

then 

(22) 

E(y.)  Z  E(yt). 

Conversely ,  i/  2?(y,)  is  finite  and  (22)  ZioZds  /or  every  ty  then  (20)  and  (21)  hold 
for  every  t. 

Proof. 

(23)  E(y.)  =  t  f  VsdP+t  [  yndP 

n  =  l  .  J  n  =  1  ,  , 

=  L  (  rfP  +  L  f  VndP 

n  =  1  .  .  J ,  ,  n  =  l  ,  J  . .  , 

{a^n,<=n}  {8*=n,f>n} 

^  Z  f  yndP  +  it  f  ytdP 

n  =  l  .  .  J J  ,  n  =  1  ,  *  , ,  , 

{8^n,t  =  n\  {*  =  n,$>n} 

=  £(».). 
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To  prove  the  converse,  for  a  fixed  n  let 

(24)  F  =  {s  >  n  and  E(y8  |$F„)  <  yn}  ; 
then  F  £  $Fn.  Define 

fs,  CO  ^  F, 

(25)  *'  =  „ 

co  £  F. 

Then  V  is  a  sampling  variable.  Since  P(?/s)  is  finite,  by  (22)  E(yn )  <  <*>  and  then 
E(yt')  exists.  But 

(26)  E{yv)  =  f  yv  dP  +  f  yv  dP  =  /  y.  dP  +  f  yn  dP 

lt'=8)  { t'  =  *} 

S  I  ysdP  +  Jv  y,  dP  =  E(y.). 

\t'  =  s} 

But  by  (22),  E(yt)  g  2?(j/«).  Hence 

(27)  fv  yn  dP  =  Jv  ya  dP 

and  therefore  P(V)  =  0,  which  proves  (20).  To  prove  (21)  let 


and  define 


V  =  {s  =  n,  t  >  n,  and  E(yt |SF»)  >  yn}, 


s,  «  $  F, 

<,  co  E  F. 


(30)  i%r)  =  f  W  dP  +  Jv  yv  dP  =  J  ysdP  +  Jy  yt  dP 
{<'=*}  {«'=*} 

^  J  Vs  dP  +  jy  yH  dP  =  J  yedP  +  JvysdP  =  E(y.), 

«'=*}  {*'=«) 

and  again  P(F)  =  0,  which  proves  (21). 

Lemma  2.  If  for  each  n, 


and  if 


E  (y  n)  =  ynf 


s  >  n, 


lim  inf  J  y£  dP  =  0, 


then  for  each  n, 


E(ys\3n)  ^  2/n, 


s  >  n. 
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Proof,  (compare  [2],  p.  310).  Let  V  £  $n  and  U  =  F{.$  2:  n}. 
Then 

Ja  yndP  =  f  y„dP  +  f  yn  dP 

V{a=n }  V[{>n\ 

g  j  yn  dP  +  J  yn+ 1  dP 


(34) 


F|»  =  nl 


V{s>n\ 


V[n  ^ s  ^n+1) 

F(ng^n+r) 

V{n  ^ 8  ^n+r} 


j  y.dP  +  f  yn+ 1  dP 

-  "•  F|«>n+1) 

J  y,  dP  +  J  2/n+r  dP 

»^n+r)  F{s>n+r} 

g  J  y,dP  +  j  y£+r  dP. 

F{n^i«^n+r}  {s>n+r} 

Therefore 

(35)  L  yndP  g  J  y,  dP  +  lim  inf  J  yZ  dP  =  f  y,  dP, 

F{8^n)  71  {8>n} 

which  is  equivalent  to  (33) . 

Lemma  3.  If  for  each 

(36)  E(yn+i\5n)  ^  yn, 
and  if 

(37)  lim  inf  f  dP  =  0, 

n  {tin} 

then 

(38)  E(yt\$n)  S  yn,  s  =  n,  t  ^  n 

Proof.  Let  V  £  $n  and  U  =  V  {s  =  n,  t  ^  n} .  Then 


s  ^  n 


(39)  jv  yndP  =  J  yn  dP  +  J  yn  dP 

=  n,t=n }  F{s  =  n,  t  >n } 

J  yndP  +  j  yn+i  dP 

t  —  n]  V\8  =  n,  t  >n } 

f  yt  dP  +  J  y„+ 1  dP 

' *  1 }  F{8  =  n( f >»4*1 } 

J  ytdP  +  J  2/n+r  dP 

F{«  =  n,  n  ^n  +  r)  V{«»n,  f  >n+r) 

J  yt  dP  —  J  yn+r  dP. 


V{8  =  n,  t=n} 

-  / 

F{«  =  n,  t —n} 

F{«  —  n,  n^t  ^n-j-1 } 

>  •  •  • 


F{«-n,ng<  ^n  +  r} 


U>n  +  r} 
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Therefore 


(40)  f  yndP  ^  \  yt  dP  -  lim  inf  f  y~  dP  =  [  yt  dP , 

n  {<>*} 

which  is  equivalent  to  (38). 

We  can  now  state  the  main  result  of  the  present  paper. 

Theorem  1.  Let  ( yni  9F«)  be  an  integrable  stochastic  sequence  in  the  monotone 
case  and  let  s  be  the  conservative  sampling  variable 

(41)  s  =  least  positive  integer.;  such  that  E(yj+i\$j)  ^  yj. 

Suppose  that  E(ys)  exists  and  that 


(42) 


lim  inf 

n 


/ 

{s>n} 


y»  dP  =  o. 


If  t  is  any  sampling  variable  such  that  E(yt )  exists  and 


(43) 

then 


lim  inf 

n 


J  y„  dP  =  0, 

U  >n\ 


(44)  E{y.)  ^  E(yt). 

Proof.  From  lemmas  1,  2,  and  3  and  relations  (5). 

We  shall  now  establish  a  lemma  (see  [2],  p.  303)  which  provides  sufficient 
conditions  for  (42)  and  (43). 

Lemma  4.  Let  (; yn ,  $n)  be  a  stochastic  sequence  such  that  E(y%)  <  <x>  for  each 
n  ^  1,  and  let  s  be  any  sampling  variable .  If  there  exists  a  nonnegative  random 
variable  u  such  that 


(45) 
and  if 

(46) 

,  then 

(47) 


E(su)  <  cc, 

E[(yn+ 1  -  yn)+ |SF»]  g  u , 

^(ys+)  <  00 ,  lim  f  yi  dP  =  0. 

n  U>n} 


S  >  71, 


Proof.  Define 

(48)  Zi  =  yt ,  *n+i  =  (t/*+i  -  t/«)+  for  n  ^  1,  w„  =  Zi  +  *  •  •  +  z„. 


Then 

(49)  ^ 

(and  hence  yt  ^  if  s  ^  n),  and  by  (46) 

(50)  ^(Zn+lifFn)  ^  W, 


s  >  n. 


100  FOURTH  BERKELEY  SYMPOSIUM:  CHOW  AND  ROBBINS 

Hence 


:dP 


(51)  E(y+)£E(w,)  =  £  I  wndP=±±  f 

n~ 1  (»  =  n|  n  — — 1(*  =  n| 

00  OO  00  J* 

=  EI  /  ZjdP  =  Y.  I  ZjdP 
J-1 

=  W)  +  E  f  E(zj\5j-i)  dP 
1  2  !«>>-u 

^  E(yt)  +  E  /  wdP  =  W)  +  E  E  (  »dP 

i  =  2  loVl)  J=2n=y  lain! 

=  W)  +  E  [  {n -l)udP  =  E{yt)  +  P(sw)  -  E(u) 

n  =  2  .  •/  . 

!«  =  ») 

g  j£(?/l+ )  +  E(su)  <  00 , 
and  hence  from  (49) 

(52)  lim  f  y£  dP  <  lim  f  w8  dP  —  0. 

n  {8>n}  71  {« >n } 

Remark.  Lemma  4  remains  valid  if  we  replace  a+  by  or  or  by  \a\  throughout. 


4.  Application  to  the  sequential  decision  problem  of  section  2 

Recalling  the  problem  of  section  2,  let  x,  xi,  x2,  •  •  •  be  independent  and  iden¬ 
tically  distributed  random  variables  with  E{x+)  <  oo ,  $n  the  <r-algebra  generated 
by  Xif  •  •  •  ,  Xnj  g(n)  ^  0,  f(n)  =  gr(n  +  1)  —  gr(n)  >  0  and  nondecreasing,  mn  = 
max  (xh  •  •  •  ,  xn),  and  yn  =  mn  —  g(ri).  The  constants  <*n  are  defined  by 

(53)  E[(x  -  a»)+]  =  /(n) 

and  are  nonincreasing;  we  are  in  the  monotone  case,  and  the  conservative  sam¬ 
pling  variable  s  is  the  first  j  ^  1  such  that  my  ^  ay;  thus 

(54)  {$  >  n}  =  {mn  <  an}. 

We  have  shown  in  section  2  that 

(55)  P{s  <  oo }  =  1 ,  E(sr)  <oo  /or  r  ^  0. 

We  wish  to  apply  theorem  L  As  concerns  s  it  will  suffice  to  show  that 
E(y?)  <  00  and  that 


(56) 


lim 

n 


j  yi  dP  =  0, 

{«>n}  - 


which  we  shall  do  by  using  lemma  4.  Let 


(57) 


Y„  =  mi  -  g(n). 
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Then 


(58)  Yt  =  y*,  E(Y+)  =  E(y+)  Z  E{xt  +  ■  ■  ■  +  z+)  =  nE(x+)  <  «o, 

and 

(59)  E[(Yn+1  -  7„)+|SF„]  =  E{[m++i  ~  m+  -  /(»)]+|ff.} 

g  E[(m++1  -  m+) |ff„]  g  E(x++l\5n) 

=  E(x+)  <  oo . 


Hence  by  lemma  4,  setting  u  =  E(x+), 

(60)  E(y+)  =  E(Y+)  <  oo 

and 


(61) 


lim 

n 


I  yi  dP  =  lim  J 

l*>n}  n  {s  >n} 


Yt  dP  =  0, 


which  were  to  be  proved. 

To  establish  the  conditions  on  t  of  theorem  1  we  assume  that  Ex~  <  oo  ;  then 
since  y «  ^  xf  +  g(n)  it  follows  that  E(yn )  <  ».  Define  a  random  variable  u 
by  setting 


(62)  u(  «)  =  /(ft)  if  t(  co)  =  ft. 

Since 


(63)  (yn+i  -  yn)~  g  f(n) 
and  /(ft)  is  nondecreasing,  it  follows  that 

(64)  E[(yn+ 1  -  tf»)“|tFn]  ^  ^  if  *  ^  n. 

We  now  assume  that  /(ft)  ^  A(ft),  where  fe(ft)  is  a  polynomial  of  degree  r  ^  0, 
and  that  E(tr+l)  <  oo .  Then 


(65) 

Since 


E(tu)  =  ± 

n  =*  1 


ft/(ft)  dP  ^  £  nfe(n)P{<  =  ft}. 

n  =  1 


(66)  P(r")  =  ±  nr+lP{t  =  ft}  <  oo, 

n  =  1 

it  follows  that  E(tu)  <  oo .  Then  by  the  remark  following  lemma  4, 

(67)  E(yr )  <  oo  and  lim  f  yn  dP  =  0, 

”  {*>«} 

and  all  the  conditions  of  theorem  1  are  established.  Thus  we  have  proved 
Theorem  2.  Suppose  that  E\x\  <  oo  and  that  in  addition  to  the  conditions  on 
g(n)  in  the  first  paragraph  of  this  section  we  have  /(ft)  ^  h(n),  where  h(n)  is  a 
polynomial  of  degree  r  ^  0.  If  t  is  any  sampling  variable  for  which  P(£r+1)  <  oo 
then  —co  <  E(yt)  ^  E(y8)  <  oo,  where  s  is  the  conservative  sampling  variable  de¬ 
fined  by  (54). 
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If  g(n)  =  nc  then/(n)  =  c  an4  we  can  take  r  =  0.  Hence 
Corollary  1.  If  E\x\  <  oo  and  yn  =  rnn  —  cn>  0  <  c  <  oo ,  then  if  t  is  any 
sampling  variable  for  which  E{t)  <  oo,  E(yt)  g  E(ye)  =  a  [see  (19)],  where  a  is 
defined  by  E(x  —  a)+  =  c  and  s  =  the  first  j  ^  1  such  that  Xj  ^  a.  Thus  s  is 
optimal  in  the  class  of  all  sampling  variables  with  finite  expectations. 

To  replace  the  condition  E{tr H)  <  oo  in  theorem  2  and  corollary  1  by  condi¬ 
tions  on  yt  we  require  the  following  theorem  which  is  of  interest  in  itself.  We 
omit  the  proof. 

Theorem  3.  Let  F(u)  be  a  distribution  function.  Define  G(u)  =  XI»=i  E(u  +  n). 
Then  G(u)  is  a  distribution  function  if  and  only  if 

(68)  Jj  u  dF(u)  <  oo , 
and  for  any  integer  b  ^  1, 

(69)  Jo  ub  dG(u)  <  oo 
if  and  only  if 

(70)  Jj  u 6+1  dF(u)  <  oo . 

Corollary  2.  If  yn  =  mn  —  cn,  0  <  c  <*> ,  and  b  is  any  integer  ^  1,  then 

(71)  #(sup  yt)h  <  oo 

n^l 

if  and  only  if 

(72)  E(x+)b+1  <  oo. 

Proof.  We  can  assume  c  —  1.  Define 

(73)  G(u)  =  P{mpy£  ^  u}. 

n  ^1 

Then  for  u  ^  0, 

(74)  G(u)  =  P{x i  ^  u  +  1,  x2  ^  u  +  2,  •  *  •  ,  xn  ^  u  +  n,  •  •  •} 

00 

=  n  +  n)- 

n  =  1 

By  theorem  3, 

(75)  E(s\ipy£)h  =  f  ubdG(u)  <  oo 

n^l 

if  and  only  if 

(76)  fj  #>  dF(«)  =  E(x+)b+1  <  oo . 

Theorem  4.  Assume  2?|x|  <  oo,  E(x+)2  <*>.  If  yn  =  mn  —  g(n)  where  g(n) 
is  a  polynomial  of  degree  r  ^  1  such  that 

(77)  flr(l)  >  0, 
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g(n  +  1)  —  g(n)  is  'positive  and  non  deer  easing,  then  for  any  sampling  variable  t, 

(78)  E(yt)  g  E(y.), 

where  s  is  the  conservative  sampling  variable  defined  by  (54). 

Proof.  By  theorem  2,  if  E(tr)  <  <*>  then  (78)  holds.  Hence  we  can  assume 
that  E(tr)  =  <» .  Now 

ff(l)  >  0,  /( 1)  =  9(2)  -  g(  1)  >  0, 

Sr(2)  £0(1)  +  /(1),  g(3)  -g(2)^f(l), 

(79)  g( 3)  g:  g(l)  +  2/(1), 


g(n)  ^  fif(l)  +  (n  -  1)/(1). 

Let 

(80)  a  =  |  min  [fir(l),/(l)]  >  0. 

Then  by  (79), 

(81)  g(n)  ^  an  for  n  ^  1. 

Let 

(82)  yn  =  mn  -  |  n. 

By  corollary  2,  E(yf )  <  oo .  Then  since 

(83)  yt  =  yt  +  ^t  -  g(t)  ^  yf  -  | g(t) 
we  have 

(84)  E(yt)  ^  £(&+)  -  ^E[g(t)]  =  -*>, 

so  that  (78)  holds  in  this  case  too. 

Remark.  If  in  the  case  ^(n)  =  cn  we  define  yn  =  xn  —  cn ,  then 

(85)  yn  ^  2/»,  2/,  =  2/s. 

Hence  for  any  sampling  variable  t , 

(86)  «®i)  ^  #(</*)  ^  #(</.)  =  «(».), 

so  that  s  is  also  optimal  for  the  stochastic  sequence  ( yn ,  $n). 

5.  A  result  of  Snell 

As  an  application  of  lemmas  1  and  2,  we  are  going  to  obtain  Sneirs  result  on 
sequential  game  theory  [3]. 

Lemma  5  (Snell).  Let  (yn,  $»)  be  a  stochastic  sequence  satisfying  yn  ^  u  for 
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each  n  with  E\u\  <  oo .  Then  there  exists  a  semimartingale  (xn,  ffn)  such  that  for 
every  sampling  variable  t  and  every  n, 

(87)  E(xt\$n\  ^  Xn  if  t  ^  n,  xn  ^  E(u\$n), 

(88)  xn  =  min  [yn,  E(xn+ i|ff«)], 
and 

(89)  lim  inf  xn  =  lim  inf  yn. 

We  will  assume  the  validity  of  this  lemma,  and  prove  the  following  theorem 
by  applying  lemmas  1  and  2. 

'  Theorem  5  (Snell).  Let  ( yn ,  Jn)  and  (xn,  $n)  satisfy  the  conditions  of  lemma  5. 
For  e  ^  0  define  s  =  j  to  be  the  first  j  ^  1  such  that  Xj  ^  yj  —  e.  If  e  >  0,  then 

(90)  E(y8)  £  E(yt)  +  e 

for  every  sampling  variable  t.  If  e  —  0  and  if  P{s  <  °o)  =  1,  then  (90)  still  holds. 

Proof.  It  is  obvious  that  in  both  cases  $  is  a  sampling  variable.  We  need  to 
verify  that  P{s  <  <»}  =  1.  If  e  >  0,  by  (89)  this  is  true. 

Since  (< xn ,  $n)  is  a  semimartingale, 

(91)  E(Xn+l\$n)  ^  Xn. 

By  (88)  and  the  definition  of  $, 

(92)  E(xn+i\$n)  =  Xn  for  s  >  n. 

Since  —  xn  ^  JS7( — ^|tFn)  and  E\u\  <  oo,  by  lemma  2  and  (92),  we  have 

(93)  E(x9\$n )  SS  xn  for  s  >  n. 

By  (87),  (93),  and  lemma  1,  we  obtain  E(x8)  g  E(xt),  and  therefore,  by  defini¬ 
tion  of  s, 

(94)  E(y8)  ^  E(x8)  +  c  ^  E(xt)  +  e  ^  E(yt)  +  e. 

Thus  the  proof  is  complete. 

J.  MacQueen  and  R.  G.  Miller,  Jr.,  in  a  recent  paper  [1],  treat  the  problem 
of  section  2  by  completely  different  methods.  Reference  should  also  be  made  to 
a  paper  by  C.  Derman  and  J.  Sacks  [4],  in  which  the  formulation  and  results 
are  very  similar  to  those  of  the  present  paper. 
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